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Abstract 


Introduction. The Internet has radically changed the global 
availability of scholarly publications. Today, a substantial part 
of the resources accessible for researchers and university 
students are offered through electronic site licenses, making the 
supply of easily obtainable information larger than ever. This 
brings forth an important question: what are the qualitative 
and quantitative effects of this development on the use of 
reference material in research and studies? 

Method. To address the research question, reference lists of 
Masters' theses from 1985,1993 and 2003 were studied in three 
disciplines: economics, psychology and mathematics, followed 
by semi-structured interviews of students who had finished 
their thesis in 2003. 

Analysis. The quantitative data were analysed using analysis 
of variance (ANOVA) with the statistical program SPSS, where 
the significance of the results was measured with Dunnet's t- 
test. 

Results. The findings showed a substantial increase in the use 
of scholarly articles as references throughout the studied time 
periods, although the amount of other information sources had 
remained largely unchanged. There were also significant 
differences between the three disciplines in the contents of their 
reference lists, the amount of Internet resources that students 
used, how they sought and obtained publications, and how they 
selected their sources. 

Conclusions. The Internet appears to have had a profound 
effect on the type and quantity ofinformation that students use 
as references in Master's theses. One of the main problems that 
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students reported was a lack of training in information seeking, 
and the abundance of irrelevant information on the Internet. 
Many respondents would have needed additional training on 
using library databases. 


change font 


Introduction 

There has been a paradigm shift in the process of scholarly communication. Until 
recently, end-users (readers) were still physically quite restricted to scholarly 
publications that could only be obtained on paper, for instance in libraries. Today, 
with the rise of the Internet, the end-user is no longer physically bound to material 
that is close by, but can easily search, obtain and download scholarly papers in 
electronic journals, electronic books, electronic archives etc. The Internet is a 
relatively new channel for scholarly resources, and contains vast quantities of 
information that vary a great deal regarding its contents, aim, target group, reliability 
etc. Hence, it is important that the end-user is aware of the diverse information 
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available on the Internet, and educated in the criteria by which the information 
content should be assessed f Chapman 2002I . 

Students have only recently received the opportunity to use the Internet to seek and 
obtain scholarly material and, consequently, knowledge on how effectively they make 
use of this channel is limited. Students' information seeking culminates as they work 
on their Masters' theses. In this sense, studying the effect of the Internet on students' 
work in this phase is particularly interesting. 

The aim of this article is to report on a study of the effect of the Internet on students' 
choice of information sources and the methods they used to find and obtain 
publications. The customs and the requirements of students when writing a Master's 
thesis may significantly differ between disciplines; therefore, this study covers 
students from three different disciplines: economics, psychology and mathematics. 
The topic is examined both on a general level, as well as individually for each 
discipline. 

The research questions that guided this study can be grouped into two sets. The first 
set was examined by studying reference lists of Masters' theses from the years 1985, 
1993 and 2003. The specific research questions concerned how use of the Internet has 
influenced the end-users choice of information sources. More specifically: Have the 
different types of references available changed in popularity? Has the number of 
references changed? and Has the average age of the references read changed? The 
second set of research questions was examined through phone interviews with 
respondents who had written their Master's thesis in 2003. More specifically: Which 
methods do ends users apply when seeking or retrieving information? To what extent 
do they use the Internet to make searches? and Which variables affected the end- 
users' choice of information sources? It was also of interest to find out what formal 
training students had received in information seeking, and whether they perceived 
their skills to be sufficient in order to find the information they needed? These 
research questions were formulated following a review of the literature, which is 
presented in the next section. 

New channels for scholarly publications 

The Internet is an attractive medium for seeking and obtaining information, for the 
following reasons: 

• The Internet is accessible twenty-four hours a day. 

• The end-user does not have to visit a library. 

• It is possible to find and obtain information relatively quickly and conveniently. 

• The end-user can choose between saving, printing or reading the information 
from the computer screen. 

• Sources on the Internet are often more up-to-date than sources in paper format. 
The Internet has made it possible to disseminate information electronically .As a 
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result, the end-user has more options in seeking and obtaining informationthan ever 
before. He or she can, for instance: borrow material from libraries, download free 
information on the Internet, or obtain information from databases on the Internet 
that Universities subscribe to (i.e., full text databases such as Proquest, Ebsco and 
Science Direct). The end-user can also obtain information by e-mail about potentially 
interesting articles through article alerting services of scholarly journals. 

The Internet has in particular affected the supply of scholarly journals to university 
students. End-users have a much larger selection of scholarly journals to choose from 
than, for example, ten years ago. The Internet has affected the supply of scholarly 
journals in the following way: 

• The supply of scholarly journals has increased. The bundling of journals as part 
of license agreements gives students the opportunity to seek and browse 
through thousands of journals. 

• The supply of other scholarly publications, such as conference proceedings, 
working papers, manuscripts etc., has also increased, as end-users can browse 
them on the Web. Scholarly journal articles can also be obtained from non¬ 
commercial, open access e-journals, or from authors' or universities' home 
pages or open archives. 

• End-users can also obtain information from companies' Web pages, news Web 
pages, communities of interest or practice etc. 

Recently, universities in Finland have gained access to large databases containing 
scholarly journals, through the FinnElib consortium. These packages include various 
databases, such as Proquest, Ebsco and Science Direct (from now on referred to as 
university databases ). Since students and the personnel of the universities are the 
only ones allowed to make use of the databases, the supply of scholarly journals has 
only increased in the universities of Finland. As a consequence of these developments, 
Hedlund (Personal Communication) argues that the amount of irrelevant information 
has greatly increased. In order not to be overwhelmed by the vast amount of 
information it is crucial that students are aware of the information sources that are 
relevant for their discipline, as the supply of scholarly journals for university students 
is now larger than ever. 

According to Chapman's (2002) study, students are often not able to make sound 
judgments about the quality and relevance of information. This maybe a problem for 
many end-users, since the quality, aim and target audience of the information may 
vary significantly. Griffiths (2003), who studied users' perceptions of JISC's (Joint 
Information Systems Committed information services, found that students are not 
very adept at evaluating the quality of online scholarly publications, since they 
seemed confused about the actual meaning of quality scholarly publications. In this 
sense it can be considered very important that students are taught to seek and 
evaluate information on the Internet (and elsewhere) effectively. 

Researchers' use of scholarly publications on the Internet 
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Many studies have been conducted regarding the type of information the end-users 
seek and obtain on the Internet and in which circumstances they prefer electronic 
sources to paper sources f Applebee et al. 1 QQ 7 : Applebee etal.. 2000 : Bjork and Turk 
2000 : Fescemever 2000 : Savolainen iqqq : Katz and Aspden 1 QQ 7 : Tenopir et al. 

200 3 )- 

Bjork and Turk (2000) studied how researchers in the domains of construction 
information technology and construction management obtained publications on the 
Internet and how they choose their sources. According to their survey, approximately 
50% of all the scholarly publications were obtained from the Internet. Fescemeyer 
f 2000k who studied geography student's use of sources on the Internet by analyzing 
citations from a test, found that 51% of the citations referred to sources on paper, 47% 
of the sources were from the Internet and the rest was course material. 

Applebee et al. (1QQ7 : 2000I studied Australian academics' use of the Internet. In 
1997 less than 50% used the Internet and only 13% used it daily. In 2000 more than 
92% used the Internet and about 25% used it daily. Applebee et al. (1997) found 
significant differences between how much people from different disciplines used the 
Internet. The respondents from the Institution of Information Technology used the 
Internet more often than others. They stated fewer problems with using the Internet, 
as compared to respondents from the other disciplines and showed more patience 
with for instance waiting while downloading information. Over 80% of the 
respondents thought that they had too little time to use the information they had 
obtained on the Internet. Over 90% said that they did not have enough time to seek 
various information sources on the Internet, which might be relevant for their field. 
This might be a sign that the respondents are having difficulties either with finding 
the appropriate methods for seeking information, or with distinguishing irrelevant 
sources. On the other hand problems in information seeking can also stem from 
having insufficient knowledge of the field of study. 

Students seeking scholarly publications on the Internet 

Several theories and models have been developed on end-user information seeking; 
among others Ellis's and Haugan's (1997) model on search patterns indifferent phases 
and types of projects, Wilson's fiQQQl problem-solving model and levels of 
uncertainty in information seeking, Kuhltau's (1993) six-stage information seeking 
model in relation to performing a task and Chu's (1999) model consisting of different 
phases in the research process with regard to researchers' information needs in each 
phase. Choo et al. (1998; 2000I combined Ellis' model of information seeking 
strategies with Aquilar's modes of organizational scanning and extended the new 
model to include information seeking on the Web. According to their study, the end- 
user can apply a variety of search methods in different stages when seeking 
information on the Internet, in order to find the information that satisfies their needs 
at the time. 

Schmidt and Spreng (1996) studied variables that influence end-users' information 
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seeking patterns. Among these are personal variables, or internal variables that vary 
among different end-users, for instance motivation, attitudes and experience 
f Schmidt and Spreng iqq6 : Kling and Callahan 2002I . as well as end-users' 
personality traits fHeinstrom 2002I . An individual's knowledge about the subject and 
of seeking information online also influences which methods he uses to find 
information fKwong 2002 : Holscher and Strube 20 oof and how successful he is in 
finding relevant information fKuo et al 2002! . In this sense, end-users who have 
received formal training in information seeking may use different methods for finding 
information than those who have not. Pennanen and Vakkari (2003) also found that 
students' ability to turn their conceptual constructs of their research proposals into 
appropriate query terms significantly affected their search success. Situational 
variables, or external variables connected to the environment, also affect individuals' 
search patterns. For instance, time limits, information needs, the supply and access of 
paper sources versus sources online and access to the Internet f Burnett and McKinley 
iqq8 : Crisp et al. 1QQ7 : Kling and Callahan 2002 : Kwong 2002 : Savolainen iqqq : 
Thong et al. 2002I . 

According to Schmidt and Spreng (iqq6I the end-user will perform an additional 
search as long as the marginal benefit of performing the search exceeds the marginal 
experienced cost. The experienced cost is described as a combination of an 
individual's access to information, degree of uncertainty and degree of pressure. 
According to Wilson ( iqqq! information seeking involves uncertainty, which 
decreases as more information is gathered. 

In summary information seeking can be described as a task-related, stepwise process, 
which is affected by both personal and situational variables, as well as an individuals' 
experience and knowledge about information seeking and the task at hand. The 
Internet provides a means to seek and obtain information suitable for information 
needs in different stages of the search process. 

Methods 

This study was made as a cross-sectional analysis, with the aim of mapping out and 
describing where and how end-users (students who are seeking information for their 
Master's thesis in three different subjects) seek and obtain scholarly publications, 
what affected their choice of methods when seeking information, retrieving 
information and their choice of sources. The study that was carried out constituted of 
two parts, which are presented in the following sections. 

Part 1: Study of reference lists in Masters' theses 

The data gathering process started in the author's own university, the Swedish 
University of Economics and Business Administration, in Helsinki, Finland, since one 
aim was to study students? information seeking and usage patterns in the university's 
library and it was expected to get a good response rate in the phone interviews in the 
second part of the study. Since all students in this university were in the field of 
economics, it was of interest to include other fields as well. The University of Helsinki, 
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which is the largest university in Finland, was chosen as another target. Two other 
subjects were chosen as comparisons to economics: psychology and mathematics. 
These fields were chosen as comparison groups because the aim was to compare 
information seeking patterns and citing patterns of students in different fields, in the 
hope of getting more reliable and generalizable results. Although the data were 
gathered from only two universities in Helsinki, the sample represents the population 
well. First, the University of Helsinki is large and the main university in this region. 
Second, it can be expected that there are greater differences in information seeking 
patterns between fields than between universities. 

In this part of the study 219 reference lists from Masters' theses were gathered from 
the subject areas of economics, psychology and mathematics in three time periods: 
seventy-one from 1985, seventy-five from 1993 and seventy-three from 2003. The aim 
was to gather twenty-five reference lists from each subject for each year, using 
random sample as the method for data selection. In some cases there were fewer than 
twenty-five available masters' theses in one year and in these cases all samples were 
included. These three time periods were chosen in order to at least partly control for 
factors, other than the effect of the Internet, which could induce changes in citing 
patterns. The year 1993 was chosen as a comparison year because this was one of the 
last years before the World Wide Web began to have effects on information 
availability. The year 1985 was chosen in order to compare changes between 1993 and 
2003 and to find out whether the changes were linear between 1985 and 2003, or 
whether changes were different or more significant between 1993 and 2003. In other 
words the aim was to find out which changes could be ascribed to the effects of the 
Internet and which could have been induced by other factors. For a summary of the 
data collected, see Table 1: 


Year 

Economics 

Psychology 

Mathematics 

Total 

1985 

25 

25 

24 

71 

1993 

25 

25 

25 

75 

2003 

25 

25 

23 

73 

Total 

75 

72 

72 

219 

Interview sample 
(2003) 

25 

25 

23 

73 

Interview participants 

15 

17 

16 

48 


Table 1: The sample 


Based on the reference lists the different types of information sources were divided 
into the nine categories below and the contents of the reference lists from 1985,1993 
and 2003 were compared. 

1. Monographs and course literature, which includes course books, compendiums 
and student?s notes from courses. 


littp://www.informationr.net/ir/12-2/paper305.html[6/21/2016 5:13:24 PM] 














Students seeking information for their Master's thesis - the effect of the Internet 


2. Scholarly journal articles. 

3. Conference publications, working papers and reports. 

4. Theses (masters', licentiate and doctoral theses). 

5. The news press, i.e., popular articles that are not academic. 

6. Grey literature, i.e., separate reports, talks, lectures and unpublished papers, 
which are not controlled by any (often commercial) publisher. 

7. Company information, i.e., brochures that describe the activities of a company, 
annual reports etc. 

8. Official sources, i.e., laws and official statistics. 

9. Other/undefined, i.e., sources whose references are not comprehensive enough 
and sources that do not fit into the categories described above (for instance 
computer programs or code). 

These sources were either in paper or electronic format, i.e., the grouping of the 
sources was based on their contents, not on the medium in which the publication was 
obtained. This was done because the use of electronic sources naturally has increased 
dramatically since 1993, as electronic sources practically were not used at all at that 
time. However, it was of interest to analyse which sources had increased or decreased 
in number between 1985,1993 and 2003 and the students' preferences for sources. 
The data were analysed with analysis of variance (hereafter ANOVA) using SPSS 
fStatistical Package for the Social Sciences! 

Part 2: Interviews 

The population, from which the interview sample was drawn, included seventy-three 
respondents who had written their Masters' theses in 2003. Of these forty-eight 
respondents took part in the interviews (making the response rate 66%); namely 
fifteen from economics, seventeen from psychology and sixteen from mathematics 
(see Table 1). Thus the distribution of respondents from the different disciplines was 
quite even. The distribution of the sexes in the sample was also even: 46% were male 
and 54% female, although it ought to be noted that all of the respondents from 
psychology were female. But since the proportion of males in the psychology 
population was two only out of twenty-five (8%) the sample represents the population 
quite well. During the first conversation a time was booked with the respondents 
when they would have access to their Master's thesis. This would help them to refresh 
their memory and thus give more accurate answers. Three of the respondents were 
abroad and preferred to answer by e-mail. 

The phone interviews addressed questions that could not be answered by analysing 
the reference lists. These included: the methods used to seek and obtain information 
and whether the user had received formal training in information seeking. Both 
quantitative and qualitative data were gathered in the interviews. Likert-scales, 
ranging from 1-5 (where 1 = never and 5 = very often), were applied to the questions 
on the methods the respondents used to seek and obtain sources. The differences 
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between the disciplines were analysed using ANOVA with the statistical program 
SPSS. More specifically, the significance of the results was measured with Dunnet's t- 
test. Since part of the aim of this paper was to map out the current situation, it was 
necessary to collect quantifiable data. In these semi-structured interviews open 
questions were also included, where the respondents were encouraged to tell more or 
specify their answers fLundahl and Skarvad iqqqI . 

The interviews with respondents from the Swedish School of Economics and Business 
Administration were conducted in Swedish, whereas those with respondents from the 
Helsinki University were conducted in Finnish, because the respondents had different 
mother tongues. Finally, all answers were translated into English. 

Findings 

In accordance with Applebee et al.'s ( Tqq7 : 2000I and Griffiths's (2003) findings 
there were significant differences between the respondents? information seeking 
methods, methods of retrieving information and the use of information sources 
between the disciplines. The data were analysed with ANOVA, in order to find out 
whether there were any significant differences between the fields and whether there 
were any significant changes in citation patterns between 1985,1993 and 2003. The 
findings are presented in the following order: 1) Methods that students used when 
seeking information; 2) methods that students used to obtain information; 3) how 
this is reflected in changes in reference lists between the years 1985,1993 and 2003 
and finally 4) students' training in information seeking. 

Information seeking methods 

During the phone interviews the respondents were asked what methods they used to 
seek and obtain information. The goal was to find out how much they used the 
Internet to find sources and how many sources they actually obtained from the 
Internet. The respondents were asked to give their answers as an integer, ranging 
from 1 to 5, where 1 = never and 5 = very often. The results are presented in Table 2. 
Significant results between individual groups (as obtained from the ANOVA) are 
marked with a star (*), in which case the marked groups differ significantly from each 
other. There are two cases, 1. and 3., where the differences between the fields do not 
stand out from the table. In these cases the analysis presented below the table 
provides further details. 


Information 
seeking method 

Average 

Economics 

Psychology 

Mathematics 

1 . Follow up 
references 

3.8 

4 . 2 * 

4 . 3 * 

2 . 9 * 

2 . Seek in library 
(goal oriented) 

3.7 

3.9 

3.8 

3.4 

3 . Subject specific 

bibliographic 

service 

3.5 

3 . 8 * 

4 . 3 * 

2 . 3 * 

4 . Browse in 

3.3 

3 . 9 * 

3.2 

2 .8* 
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library 



5. Received source 

2.8 

2 .1* 

2 .8* 

3.5* 

6 . Follow 
hyperlinks 

2.7 

2.3 

3.1 

2.8 

7. General search 
engine 

2.6 

3.0 

2.3 

2.5 

8 . Hints from 
others 

2.6 

2.4* 

2.4 

3.1* 

9. E-mail alerting 
services 

1.3 

1.1 

4,3* 

1.1 


Table 2: Methods of seeking information 


The results are presented ranging from the most popular to the least popular method. 
First the average results are presented, where after the differences between the fields 
are described in more detail: 

1. The most popular method for finding information was to follow up references in 
other publications (see l with 3.8 in the table above). Many respondents claimed 
this method to be especially useful in the beginning, when they were not yet 
familiar with the literature of their subject field. A few respondents said that 
following references is easy on the Internet, because some references are linked 
to the cited papers. In this case, although respondents from psychology looked 
up more sources from reference lists (4.3), as compared to the other fields, the 
only statistically significant difference was between economics (4.2) and 
mathematics (2.9). This finding might be explained by the fact that the nature 
and contents of the Masters' theses in mathematics differs from that in 
economics and psychology. Whereas the aim in economics and psychology 
theses is to contribute to and extend existing research, Masters' theses in 
mathematics can focus on problems that have been solved before. This because 
in some theses in mathematics students need to prove they have the ability to 
solve a complex mathematical problem, rather than contributing to existing 
streams of research. A respondent from economics said: 'I always looked into 
articles' and books' reference lists. That was one of the best ways of finding 
more sources'. 

2. All of the respondents sought information in the library in a goal-oriented 
fashion, using the author's name, the title or the subject (see 2. with a score of 
3.7 in Table 2). There were no significant differences between the fields, 
indicating that the use of library services has remained popular among students, 
despite the growing availability of information on the Internet. 

3. It was also common among the students to seek information sources through 
subject specific portals or search engines. These include so-called open access 
sources (which are free of charge to the end-user) and databases with articles 
from scholarly journals from commercial publishers (like Ebsco, Psychlit), 
which are not free of charge but paid for by the Universities (often by licence), 
see 3 with a score of 3.5 in the table above. There were some differences 
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between the fields in the use of this method for finding information; 
respondents from mathematics used this method the least (2.3), i.e., less than 
those from economics (3.8) and psychology (4.3), whereas the respondents from 
psychology used this library service most. Some respondents from psychology 
and economics commented that it was unnecessary to use general search 
engines on the Internet, because it was much easier to find good articles 
through the databases subscribed to by the University. 

4. Although it was more popular among students to make goal-oriented searches 
in a library (see 2 in the table) instead of just browsing through information 
(either by being physically present in the library or by searching a database that 
covers the library's contents), this method was still quite common (see 4 with a 
score of 3.3). Respondents from economics relied on this method the most (3.9) 
and significantly more than those in mathematics (2.8). Some respondents in 
economics and psychology reported that they visited their university's library 
regularly to browse through new journals or to get new ideas. 

5. Some respondents received sources directly from professors or friends (see 5 
with 2.8). Whereas many respondents in mathematics received most of their 
sources from their professors or friends (3.5), this was far less common in 
economics (2.1). 

6. Although some respondents did not like to use general search engines on the 
Internet, because the searches yielded so many results, some spent a lot of time 
following hyperlinks on the Internet (see 6 with a score of 2.7). There were no 
significant differences between the fields in using this method for seeking 
information and many of those who did use it often, did so in connection with 
following up references. Yet, many respondents complained that they thought 
that searching this way was too troublesome and took up too much of their time, 
since some of them could end up following different links all day and finally only 
end up with a few relevant information sources. 

7. It was not as common to seek information using general search engines on the 
Web as one might have expected (see 7 with a score of 2.6 with no differences 
between the groups). Many of the respondents only looked for information in 
the Universities' databases, because they experienced it as difficult to find 
reliable scholarly publications by using for instance Google or Altavista. In 
contrast, two of the respondents had looked for almost all of their sources on the 
Web, because there were not enough relevant sources on their subjects in the 
subscription databases. Two psychology students commented: 'I used this 
method in the beginning when I didn't know much about the topic, to get hold 
of background information. But I did not include these sources later on in my 
Master's thesis', and 'There was so little valid information on the Internet. I 
rather searched Psychlit [the university's database] with the use of key words'. 

8. One of the least used methods for seeking information was to look for sources 
based on advice from friends, professors or experts, or by simply remembering a 
source (see 8 with 2.6). Respondents in mathematics received significantly more 
advice and hints about relevant information sources (3.1) than did respondents 
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from economics (2.4). A few respondents from mathematics even claimed that 
they had found all their sources based on their supervisors' recommendations, 
or that they simply had received all sources directly from them. 

9. It was least common to use alerting services for articles in scholarly journals 
(see 9 with 1.1). This method was used extremely rarely. Only two respondents 
from economics had used alerting services and both said that they had not 
found many relevant articles this way. As they advanced in their research 
process, however, they claimed that this service did provide some additional 
information. This result resembles Choo et al.'s 12000! study, where 
respondents also used alerting services scantily. This result might be explained 
by the fact that students who are writing their Master's thesis work quite 
intensively for a short time, whereas for instance PhD students and researchers 
want to keep up with development in their field(s) and thus may well find this 
method more useful. 

Use of the Internet and methods for obtaining information 

Comparing the share of sources obtained from the Internet to the amount of paper 
sources was interesting. A problem emerged from the manner in which some sources 
were referred to in the reference lists. Students are required to include the Web 
address of a source that has been obtained from the Internet, but this only applies to 
sources that cannot be found in paper format. For example, students are not required 
to include the Web address of an article that has been downloaded from one of the 
databases subscribed to by the University. In this sense the analysis of the extent to 
which electronic sources appear in the reference lists suffered and the true number of 
electronic sources remained unclear. Thus, this issue was addressed in the telephone 
interviews and the respondents were asked to estimate, expressed as a percentage, the 
sources they had obtained from the Internet. 

It was found that the respondents had not included all the Internet addresses of the 
sources they cited and that they had obtained more sources than presented in the 
earlier results. Based on the citations in the Masters' theses few sources referred to 
information on the Internet (approximately 8%). Based on the respondents' own 
estimations, however, approximately 32% of the sources were obtained from the 
Internet. The respondents from economics obtained most publications from the 
Internet (44%), whereas the others obtained less (psychology 28% and mathematics 
25%). The share of sources the respondents obtained from the Internet was far less 
than found, for instance, in Fescemeyer's (2000) study, where the students obtained 
47% of their sources from the Internet. 

The following table presents the different methods the students used for obtaining 
information. The methods are presented in order of precedence, ranging from the 
most popular to the least popular method. Significant differences between the 
disciplines are marked with a star (*) and are discussed below. 
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Methods of 
obtaining 
information 

Average 

Economics 

Psychology 

Mathematics 

University 

library/institution 

4 

3.4* 

4.7* 

3.8* 

University's 

database 

2.7 

3.3* 

3.6* 

1.3* 

Publisher's home 
page 

1.7 

1.9 

1.4 

1.8 

Author's home 
page 

1.4 

1.5 

1.4 

1.4 

Library distance 
loan 

1.4 

1.5 

1.3 

1.3 

Ask for paper copy 

1.3 

1.3 

1.4 

1.2 

Ask for electronic 
copy 

1.1 

1.0 

1.1 

1.1 


Table 3: Methods of obtaining information 


The respondents obtained most of their information sources from University libraries 
or University institutions, either by borrowing or copying them (average answer was 
4, see table 3). This method was most popular among respondents in all of the 
disciplines, although respondents from psychology (4.7) used this method 
significantly more than those in economics (3.4) and mathematics (3.8). The 
respondents also obtained many publications from full-text databases that the 
Universities subscribed to (2.7). Respondents in psychology used this method more 
often (3.6) than did the respondents from economics (3.3), whereas respondents from 
mathematics used it the least (1.3). Generally the respondents were satisfied with 
their full-text databases and thought that it had helped them find many relevant 
sources. In was not uncommon to retrieve sources from publisher's home pages for 
free (1.7). Respondents mostly obtained these free sources from universities' home 
pages or from open access e-journals or databases. Some respondents said that 
information on universities' home pages is more reliable than other information on 
the Web. Two less popular methods were to obtain information from authors' home 
pages or to order it from the Internet, through inter-library loans (1.4). Some 
respondents commented that they though that this method took up too much of their 
time or was simply too cumbersome. The respondents did not ask for many sources 
from authors, neither electronic copies (1.1) nor paper copies (1.3). Some said that 
they did not want to bother the authors, or they thought it took up too much time. 
Others felt that they should take responsibility of obtaining relevant sources 
themselves. On average, respondents who had received some formal training in 
information seeking obtained more publications from commercial publisher's home 
pages. 

Findings from the reference lists 

The study of reference lists between the years 1985,1993 and 2003 showed significant 
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changes between 1993 and 2003, (i.e., when the Internet started to have an effect) in 
the following ways: the number of sources in reference lists has increased 
dramatically, the average age of sources has decreased and the types of sources used 
have changed. These findings will be discussed in the following sections. 

Number of sources in the Masters' theses 

The average number or sources has significantly increased as an effect of the Internet, 
being 30 sources in 1985, remaining at 30 in 1993 and increasing to 47 in 2003. When 
studied separately the number of sources has only increased significantly in 
economics and psychology between 1993 and 2003, while remaining quite constant 
between 1985 and 1993: from c.25 in 1985 and 27 in 1993 to c.45 in 2003 and from 
c.54 in 1985 and 53 in 1993 to c.79 in 2003, respectively. The number of references 
has not increased significantly in mathematics (being c.13 in 1985,10 in 1993 and 16 
in 2003), which might be due to the fact that the contents, purpose and structure of 
Masters' theses in mathematics differ from the two other fields. See the following 
figure: 
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Figure 1: Number of sources in Master's theses 

Although the number of references in Masters' theses has increased as an effect of the 
Internet, the number of pages in Masters' theses has remained quite constant, (being 
70 pages in 1985, 55 in 1993 and 58 in 2003). There was a significant decrease in the 
number of pages in Masters' theses between 1985 and 1993, although the Internet did 
not induce it. In other words the respondents referred to more sources, without 
increasing the length of the Masters' theses, see figure 1. 

Average age of sources 

On average, the age of the sources read has significantly decreased, probably partly as 
an effect of the Internet. The average age of sources was 12.0 years in 1985,12.8 years 
in 1993 and 9.9 in 2003. Individually the age of the sources read has only decreased 
in mathematics (from 17.2 and 18.6 to 12.0 years) and psychology (from 11.2 and 12.8 
to 9.8), whereas it has slightly increased in economics (from 7.5 and 7.0 to 7.9 years). 
The decrease in the average age of the sources is not substantial, but has remained 
quite stable over the past 20 years. These results resemble Tenopir et ah 's (2003) 
study of scholarly journals, where the age of the sources also remained relatively 
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constant over a period of approximately to years. One reason for the small change in 
the age of the sources may be that the literature in the disciplines has not evolved 
rapidly and classic publications are still regarded as relevant. 

The percentage and number of different types of sources 

It was found that the Internet had affected the student's choice of different types of 
information sources for their Masters' theses. First, the changes in the percentages of 
different source types in the reference lists are presented, where after the changes in 
the number of different sources are presented. Comparisons are made between the 
disciplines. The most pronounced changes in the percentages of different source types 
in the reference lists are presented in Table 4 below, ranging from the most common 
to the least common source types cited. Significant differences are marked with a star 
(*). Note that there was no difference in the use of sources between 1985 and 1993, 
i.e., all stars refer to differences between 1985 and 2003 or between 1993 and 2003. 
Below the table follows a discussion about the differences between the disciplines' use 
of sources. 


Types of sources (%) 

1985 

1993 

2003 

Scholarly journal articles 

23.2* 

27.9 

37.3* 

Monograph/Course literature 

60.4* 

58.0* 

45.8* 

Company information 

0 .1* 

0.5* 

2 .0* 


Table 4: Percentage of different source types 


There were significant variations between the types of sources that were used in the 
three studied fields. Psychology students used more journal articles (on average 55% 
of the total number of sources), than those from economics (21%), whereas students 
in economics used more articles than those from mathematics (13%). On the other 
hand respondents from mathematics used most monographs and course literature 
(79%), whereas respondents from economics used less (48%), and psychology 
students used the least (37%). Although these were the most significant findings, 
some other differences between the fields appeared as well: Students from economics 
had more thesis citations (Masters' theses, licentiate theses and doctoral 
dissertations) than did those from mathematics (2.6% vs. 0.9%). They also cited the 
news press (i.e., non academic papers), grey literature, company information and 
official sources (8.2%, 4.7%, 2.3% and 6.9% respectively) more often than did those 
from psychology (0.8%, 1.6%, 0% and 0.3% respectively) and mathematics (1.1%, 
1.2%, 0.3% and 0.4% respectively). These differences in the citation patterns between 
the fields can probably be explained by the fact that the nature, structure and 
demands of Masters' theses differ between fields. It is no surprise that economics 
students, for instance, rely more heavily on official sources, the news press, and 
company information, since these sources may provide information about the 
business field or company of interest. Psychology students, on the other hand might 
rely more on clinical studies cited in prestigious journals, whereas those from 
mathematics might derive their mathematical formulas and basic theories from 
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monographs and course material. 

Despite the differences between the three fields, the most significant affect of the 
Internet was found to be citations to articles from scholarly journals. As mentioned 
earlier the supply of articles from scholarly journals has increased among researchers 
and students, as they now via the Internet can access full-text databases such as 
Proquest and Ebsco that aggregate publications from thousands of journals. This 
increased supply seems, in view of the empirical results, to have significantly affected 
the student's choice of information sources for their Masters' theses. The figure below 
presents the changes in the number of references between 1985,1993 and 2003: 
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Figure 2: Changes in the different types of sources, 1985 to 2003 

As illustrated above, the average number of articles from scholarly journals has been 
9.6 in 1985,11.5 in 1993 and has increased to 23.5 in 2003. The number of 
monographs and course literature in the reference lists has remained quite stable 
during the last 18 years, as have the other sources. This is not surprising, since 
students still frequently use the library (see methods for retrieving information) and 
the Internet has not affected the availability of monographs or course literature. 
Rather, the Internet has mostly increased the supply of available articles from 
scholarly journals, which, according to the findings from the interviews, are used a 
great deal by the students. It might have been expected that the number of grey 
literature sources would have increased, as they are readily available on the Web. 
This, however, is not the case. Some respondents said that they did seek background 
information on subjects through general search engines on the Internet, but that the 
quality and reliability of these sources was not adequate enough for them to be 
included in the reference lists. They stated further that these types of sources gave 
them a good insight into their subjects, especially in the beginning of the research 
process. 

If the disciplines are studied separately it is clear that the respondents from 
psychology used most articles from scholarly journals (24.2 in 1985, 27.8 in 1993 and 
55.1 in 2003), as compared to those in economics (4.6 in 1985, 4.9 in 1993 and 11.2 in 
2003) and mathematics (1.3 in 1985,1.7 in 1993 and 2.7 in 2003). The increase in the 
number of articles from scholarly journals was only significant in psychology and 
economics. Again, this might be due to the fact that Masters' theses in mathematics 
differ in purpose, structure and content from those in economics and psychology. 

Training in information seeking 
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The respondents were asked whether they had received any formal training in 
information seeking, in order to find out if it affected how students searched, 
gathered and chose their sources. About half (48 %) of the respondents had attended 
a course in information seeking and 39 % of the respondents said that they had not 
received enough training. Some respondents were unhappy with the timing of the 
training and though that more information about information seeking ought to be 
given in connection with a seminar or a course in methodology. One respondent from 
economics commented: 'The training was sparse. I have learned everything myself 
through trial and error. I received some information in the beginning of my studies, 
but nothing after that.' 

It was found that respondents who had received some formal training in information 
seeking obtained more sources from the Internet on average. They also cited fewer 
course literature sources than respondents who had not received any training. But 
this might also be dependent upon the fact that the respondents from mathematics 
had received least training and used the largest amount of course literature. 
Respondents who had attended a course in information seeking generally also used 
subject specific search engines and portals and the Universities' databases to find 
relevant information. But these differences might also be ascribed to the fact that 
respondents in mathematics received least training, used the largest amount of course 
literature and used subject specific search engines and portals the least to find 
relevant information. 

One finding, which shows the importance of teaching students different methods for 
seeking and retrieving information, is that many psychology students believed that 
full text articles could only be attained through computers in the university's library 
and as a result did not use the library's databases much for seeking or retrieving 
information from home. In fact students have the possibility of logging on to the 
university's network from home and thereby getting accesses to the full text articles. 
Problems or barriers the respondents often mentioned against effective information 
seeking included, among others: 

• Difficulties in articulating queries for search engines. 

• General dislike of using computers and lack of experience. 

• Information overload: One search on the Internet can provide hundreds of 
sources of which perhaps just 1 % is relevant. 

• Availability: Students in psychology felt that access to full-text articles in the 
subscription databases was too limited. 

Students' opinions of the Internet 

The respondents had quite different opinions about information on the Internet. 
Respondents who were not fond of using computers had, quite naturally, a less 
positive opinion about electronic information sources as compared to those who also 
surfed the Internet often for other purposes. Respondents in psychology and 
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economics were quite pleased with the academic resources that were offered by the 
universities' libraries. On average, the respondents were not as fond of searching the 
Web through general search engines, as they did not think they could find quality 
material there and the searches took too much time. There were, however, a few 
respondents who said that the information on the Web was of crucial importance for 
them, as not much had been published on paper relevant to their subject field. 

Some respondents mentioned difficulties when seeking on the Internet. Many said 
that it is hard to find relevant sources, as a search engine can produce thousands of 
results. Many pointed out that it is important to be very critical regarding sources on 
the Internet, as their origin can be quite obscure, although some thought that it is 'up 
to you' to successfully use the Internet when seeking sources. A respondent 
commented: 

'If you're good at searching you can find pretty good sources. It's up to 
yourself if you can pick and choose [among the information sources]. 

Most of my sources are from famous authors. They are more 
trustworthy and you know what types of sources they are.' 

Even though the respondents used the Internet often to seek and get hold of 
information, it seemed as though electronic sources were used more as a complement, 
in contrast to a substitute, to sources in paper format, in accordance with the findings 
of Savolainen fiQQQl . A respondent commented: 

Ninety peercent [of the information you find on the Internet] is totally 
useless when you search it using key words, most of it is useless... It is 
good as a complement for library services, but cannot replace it. 

Some respondents said that they prefer to read their sources in paper format, whether 
they have obtained it from the Internet or not. In summary it can be stated that 
electronic sources are not a threat to sources in paper format, but indeed mostly used 
to complement paper sources. 

When you obtain sources from the Internet and print them out you get 
such large amounts of paper that it gets hard to keep track of the most 
important paragraphs and texts. In books it is easy, you just put a 
post-it note between the pages 


Summary and conclusions 

The findings showed significant differences among the disciplines regarding the 
methods used to find information sources, although the respondents used very 
similar methods to get hold of publications: the majority of the information sources 
are still obtained from a physical library, even though the respondents like to seek 
information on the Internet. Respondents in economics and psychology obtained 
more sources from the Internet, whereas those from mathematics did not use the 
Internet as much for this purpose. The respondents in mathematics also cited the 


littp://www.informationr.net/ir/12-2/paper305.html[6/21/2016 5:13:24 PM] 



Students seeking information for their Master's thesis - the effect of the Internet 


smallest number of sources (sixteen, on average), whereas those in psychology cited 
the most (79, on average). These differences might be explained by the different 
research traditions of the disciplines and their demands on the quality and structure 
of Masters' theses. The most central results are, in summary: 

• The average number of sources in reference lists of Masters' theses has 
increased between 1993 and 2003, in the disciplines of economics and 
psychology. The number of pages in the Masters' theses has not increased in the 
same period. In other words electronic sources are not substituting other 
sources, but rather complementing them. 

• The average age of the sources in the reference lists has decreased in the 
disciplines psychology and mathematics. Specifically, the age of sources in the 
reference lists has decreased as an effect of the Internet, as sources on the 
Internet generally are more up to date in comparison to those in paper format 
f Chapman 2002I . 

• The number of articles from scholarly journals has increased significantly 
between 1993 and 2003, in economics and psychology. This effect might be 
ascribed to the fact that the availability of articles has increased dramatically for 
students and researchers, who have obtained access to full-text databases such 
as Ebsco, Psychlit etc. 

• The number of other types of sources has remained quite stable. This might be 
explained by the fact that the Internet has most significantly increased the 
availability of electronic articles from scholarly journals. 

• Formal training did not affect the information seeking methods the 
respondents used or how they obtained their sources. But respondents who had 
received some training did use more articles from scholarly journals and less 
course literature. 

• The respondents were not satisfied with the amount of training in information 
seeking they had received from their university. There seems to be a clear need 
for more training on information seeking. Many respondents were irritated by 
the fact that they had received some training in the beginning of their studies, 
but not much afterwards. Some suggested that training could be given in 
connection with methods courses or when writing a term paper or Master's 
thesis. Education concerning technical search methods is not sufficient, as many 
respondents had problems with dismissing irrelevant information when 
searching, forming good queries and evaluating the relevance of publications. 
Training and information about how to be critical when selecting sources and 
how relevant material can be found is also important (i.e., information literacy). 

In summary, it is quite common that end-users search the Internet or online 
databases in order to find relevant information, thus they need to be taught how to 
make effective searches and how to evaluate information on the Internet. There has 
been a general concern regarding the quality and reliability of information on the 
Web and how skilled students are at evaluating these sources. This study shows that 
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students are sceptical towards information found on the Web, for instance by using 
search engines like Google or Altavista and generally do not include many of these 
types of sources in the references of their Masters' theses. The availability of scholarly 
journal articles has increased as students and faculty have received access to full text 
databases, such as Proquest and Ebsco. The number of scholarly journal articles in 
references has increased dramatically, showing that students make good use of these 
high quality academic resources. 

As the availability of different types of information has (and is) increasing, as an effect 
of the Internet, it is important that students are taught to seek information on the 
Internet, use the subscription databases and evaluate sources. The fact that many 
psychology students assumed that full text articles could only be accessed through 
computers in the university's library and thus did not search for them from home, 
shows that an increase in available information as such (through the Internet or 
elsewhere) does not increase usage. In order to increase the effectiveness of end- 
users' information seeking they also need to be made aware of all the different 
methods available for seeking and retrieving information. 

The major finding of this study was that the citing patterns of students (when writing 
their Master's thesis) have changed to include more articles from scholarly journals, 
because the Internet provides students (and other academics) with more sources than 
any single library has before, which also are quite easy to access. Although the 
findings were quite clear there are some limitations that should be taken into 
consideration. First, other factors might have affected student's choice of information 
sources. As Finnish universities are aiming at internationalizing their research, 
students might be encouraged to use more information from international journals. 
The quality requirements of Masters' theses might also have increased between 1993 
and 2003, pushing students to obtain information from international journals that 
are highly regarded. Conversely, the access to large amounts of scholarly journals, 
provided by the Internet, might also have heightened quality requirements, since it is 
now much easier for students to obtain high quality information sources. 

Nevertheless, the causal relationships are unclear and were not studied in this article, 
which only provides evidence of an increased usage of articles from scholarly journals. 
Other studies could address this limitation by studying the factors that have affected 
citation patterns in different subjects. Second, in this paper only students (who were 
writing their Master's thesis) were included, so information seeking and citation 
patterns found in this study might not necessarily be generalizable to other 
information users, for example academics or industrial researchers. Other studies 
could include respondents from other categories as well, in order to shed some light 
on the similarities and differences in information seeking behaviour between different 
groups of people. 
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